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Abstract 
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1 Introduction 

Since Klee and Minty [1] showed that a simplex method for linear programming is not 
a polynomial algorithm, polynomial complexity bound has become a popular metric 
to measure the efficiency of optimization algorithms. Searching for polynomial al¬ 
gorithms for linear programming was a major research area of optimization between 
1980’s and 1990’s after Khachiyan [2] announced the first polynomial algorithm for 
linear programming. Although Khachiyan’s algorithm was shown to be much less ef¬ 
ficient in practice than the simplex method [3], Karmarkar’s interior-point method [4] 
demonstrated the possibility of existence of efficient polynomial algorithms. For feasible 
starting point, people quickly established polynomial bounds for various interior-point 
algorithms [5, 6, 7, 8, 9, 10]. The lowest bound of these algorithms is 0{\fnV) which 
has not been improved for more than two decades. 

To have an efficient implementation for interior-point algorithms, Mehrotra [11] and 
Lustig et. al. [12] realized that higher-order method and infeasible starting point are 
two necessary improvements. However, algorithms with either one of these features had 
poorer complexity bounds than 0(y/nL). Monteiro, Adler, and Resende [13] showed 
that a higher-order algorithm starting from a feasible point has the polynomial bound 
0(nL). For infeasible-interior-point method, Zhang [14], Mizuno [15], and Miao [16] 
established polynomiality for several different algorithms (none of them is a higher-order 
algorithm). The best complexity bound 0{nL) for infeasible interior-point methods has 
not been changed since the eary of 1990’s. 

Recently, Yang [17, 18] showed that for a higher-order interior-point method starting 
from a feasible point, the polynomial bound can be improved to 0(y/nL ) by using an 
arc-search method. Very recently, Yang et. al. [19] used the same idea and proposed 
a polynomial arc-search infeasible-interior-point algorithm with a complexity bound of 
0(n*L). In this paper, we show that for higher-order infeasible-interior-point method 
using arc-search, the polynomial bound can be improved to 0(nL), which is a bound 
at least as good as the best bound of existing infeasible-interior-point algorithms. 

The remainder of the paper is organized as follows. Section 2 describes the problem. 
Section 3 provides an infeasiblc-predictor-corrector algorithm. Section 4 proves its 
polynomiality. Section 5 summarizes the conclusions. 

2 Problem Descriptions 

The standard form of linear programming in this paper is given as follows: 


min c T x, subject to Ax = b, x > 0, 


( 1 ) 


where A G M mxn , b G M m , cgR" are given, and x G 1" is the vector to be optimized. 
Associated with the linear programming is the dual programming that is also presented 
in the standard form: 


max b T y, subject to A r y + s = c, s > 0, 


( 2 ) 
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where dual variable vector y G M m , and dual slack vector s G M n . We use S to denote 
the set of all the optimal solutions (x*,y*,s*) of (1) and (2). It is well known that 
x 6 R" is an optimal solution of (1) if and only if x, y, and s satisfy the following KKT 
conditions 


Ax = b, 

(3a) 

A x y + s = c, 

(3b) 

IV 

o 

(3c) 

i — 1,..., n. 

(3d) 


To simplify the notation, we will denote Hadamard (element-wise) product of two 
vectors x and s by x o s, the element-wise division of the two vectors by s _1 o x, or 
x o s -1 , or | if min |sj| > 0, the Euclidean norm of x by ||x||, the infinite norm of x by 
||x||oo, the identity matrix of any dimension by I, the vector of all ones with appropriate 
dimension by e, the block column vectors, for example, [x r , s T ] T by (x, s). For x G M”, 
we will denote a related diagonal matrix by X G M™ xn whose diagonal elements are the 
components of the vector x. Finally, we define an initial vector point of a sequence by 
x°, an initial scalar point of a sequence by /i o, the vector point after the Mil iteration 
by x fc , the scalar point after the Mh iteration by /ifc. Let 

r k = Ax fc - b, (4a) 

r k c = A T y fc + s k - c. (4b) 

Given a strictly positive current point (x fc ,s fc ) > 0, the infeasible-predictor-corrector 
algorithm is to find the solution of (1) approximately along a curve C(t) defined by the 
following system 


A x(t) — b = tr k (5a) 

A T y (t) + s(t) - c = tr k (5b) 

x(t) o s (t) = tx. k o s k (5c) 

(x(t),s(t)) > 0, (5d) 


where t G (0,1]. As t —> 0, (x(t), y(t), s(t)) appraches the solution of (1). Since C{t) 
is not easy to obtain, we will use an ellipse £ [20] in the 2 n + m dimensional space to 
approximate the curve defined by (5), where £ is given by 


£ = {(x(a), y(a), s(a)) : (x(a), y(a), s(a)) = acos(a) + b sin (a) + c}, (6) 

a G M 2n+m and b G R 2n+m are the axes of the ellipse, and c G R 2n+m is the center of 
the ellipse. Taking the derivatives of (5) gives 


0 

0 


X 


yk 

L b 

A T 

I 


y 

= 

r k 

1 c 

0 

x fc 


s 


h h 

X o s 
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( 8 ) 



X 


0 


y 

= 

0 


s 


—2x o s 


A 0 0 

0 A T I 

S k o x fc ^ 

We require the ellipse to pass the same point (x A: , y k , s k ) on C(t) and to have the same 
derivatives given by (7) and (8). The ellipse is given in [17, 18] as 

Theorem 2.1 Let (x(ct), y(a), s(ct)) be an arc defined by (6) passing through a point 
(x, y,s) G £ fl C{t), and its first and second derivatives at (x, y,s) be (x, y,s) and 
(x, y,s) which are defined by (7) and (8). Then, the ellipse approximation of (5) is 
given by 

x(a) = x — xsin(a) + x(l — cos(a)). (9) 

y(a) = y - y sin(a) + y(l - cos(a)). (10) 

s(a) = s — s sin (a) + s(l — cos(a)). (11) 


3 Infeasible predictor-corrector algorithm 

We denote the duality measure by 


T 

X S 


T = 


n 


and define the set of neighborhood by 

N{9) := {(x, s) | (x, s) > 0, ||xos —/ie|| < 9p}. 


( 12 ) 


(13) 


The proposed algorithm searches an optimizer along the ellipse while staying inside 

M(6). 

Algorithm 3.1 

Data: A, b. c , 9 E (0, 2+ V^ ] ; e > 0, initial point (x°,y°,s 0 ) G M{9). 
for iteration k = 1,2,... 

Step 1: If 


Lk Si ( ■ 

(14a) 

Ax fc - b|| < e, 

(14b) 

+ s k ~ c|| < e, 

(14c) 

1*T 

(/I 

V 

o 

(14d) 


stop. Otherwise continue. 

Step 2: Solve the linear systems of equations (7) and (8) to get (x, y, s) and 
(x,y,s). 
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Step 3: Find the smallest positive a £ (0,7r/2] sitch that for all a £ (0,a], 
(x(a), s(a)) > 0 and 

||(x(a) o s(a)) - (1 - sin(a))/i fc e|| < 20(1 - sin (a))/i k . (15) 

Set (to simplify the notation, we use a in stead of a in the rest of the paper) 
(x(a),y(a),s(a)) = (x. k ,y k ,s k ) - (x, y, s) sin(a) + (x, y, s)(l - cos(a)). (16) 

Step 4 : Calculate (Ax, Ay, As) by solving 


1 

o 

o 

< 

1_ 


' Ax ' 


0 

0 A T I 


Ay 

= 

0 

_ S(a) 0 X(a) _ 


As 


(1 — sin(ct))/i fc e — x(a) o s(a) 


Update 

(x‘+\ y*+i, s‘ +1 ) = (x(a),y(a), s(a)) + (Ax, Ay, As) (18) 

and 

-y-fc+D 0^+1 

fik+1 = -• (19) 

n 

Step 5: Set k + 1 —^ k. Go back to Step 1. 

end (for) 

In the rest of this section, we will show (1) —> 0, —> 0, and /i k —> 0; (2) there 

exist a £ (0,7t/ 2] such that (x(a),s(a)) > 0 and (15) holds; (3) ( x. k ,s k ) £ JV(9). It is 
easy to show that r k , r k , and fi k decrease at the same rate in every iteration. 

Lemma 3.1 

r 6 +1 = r fe(! - sin (a)), r^ +1 = rj?(l - sin(a)), fi k+l = pt, k ( 1 - sin(a)). (20) 


Proof: Using (4), (18), (16), (8), and (7), we have 

r k+i _ r k = A ( x fc+1 _ x fc) = A(x(a) + Ax - x fc ) 

= A(x A: — xsin(a') — x fc ) = — Axsin(a) = — r k sin(a). 

This shows the first relation. The second relation follows a similar derivation. From 
(17), it holds that (Ax) T As = (Ax) T (—A T Ay) = —(AAx) T Ay = 0. Using (18), we 
have 

x.fe+i T s fc+i _ ( x (q,) _|_ Ax) t (s(ck) + As) = x(cx) t s(q() + x(ck) t As + s(ck) t Ax 
= x(a) T s(a) + (1 — sin (a))p k n — x(ck) t s(q;) = (1 — sin(o;))/Xfcn. 

Dividing both sides by n proves the last relation. 
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Clearly, if sin(a) = 1 (a — |), we will find the optimal solution (allowing some 
Xi = 0 and/or Sj = 0) in one step, which is rarely the case. Therefore, from now on, we 
assume a G (0, |). We will use the following lemma of [16]. 

Lemma 3.2 Let (Ax, As) be given by (17). Then 


|Ax o As|| < -(-H(X(a')S(a')) “(x(a) o s(a) - fi l+ i e )l| 2 - 


( 21 ) 


Theorem 3.1 Let (x(a), y(a), s(ck)) and (x fe ,s fc ) G Af{9). Then, for all k > 0 

(i) there is an a > 0, such that (x(a),s(a)) > 0 and (15) holds. 

(ii) if 9 < ^ 75 , then (x fc+1 , s fc+1 ) G Af(9) for all the iterations. 

Proof: Using 1 — cos(a) < 1 — cos 2 (a) = sin 2 (a) and (x fc ,y fc ,s fc ) G Af{9), we have 

||x(cd) o s(a) — (1 — sin(o))/ifce|| 

= \\(x k o s k — /i fc e)( 1 — sin(a)) + (x o § — x o §)(1 — cos(ck)) 2 


—(x o s + x o s) sin(a)(l — cos(ck))|| 

< 9nk{l — sin(o)) + (||x o §|| + ||x o §||) sin^a) 
T(||x o s || + ||x o §||) sin 3 (a). 


( 22 ) 


Clearly, if 
q(a) : = 


xos + xos |sin 4 (a)+^ xo§ + xos j sm 3 (a)+9/ik sin(a;) — 9^ < 0, 

(23) 

then, (15) holds. Indeed, since g(0) = —9/j, < 0, by continuity, there exist a > 0 such 
that (23) holds. This shows that (15) holds. From (15), we have 

Xi(a)si(a) > (1 — 2#)(1 — sin(a))/Xfc > 0, V0G [0,0.5) and VaG[ 0 , 7 r/ 2 ). 

This shows (x(a), s(co)) > 0. Therefore, we finish part (i). Furthermore, from Lemma 
3.1, (15) is now equivalent to ||x(a)os(a) — /i fc+1 e|| < 2 9fi k+ i. Using (18), (17), Lemmas 
3.1 and 3.2, and part (i) of this theorem, we have 

||x fc+1 ° s fc+1 - (i k+ 1 e\\ 

= || (x(a) + Ax) o (s(a) + As) - ia k+1 e\\ 

= ||Ax o As|| < ^||(X(a)S(a))“3( x (a)os(a) - fi k+1 e)\\ 2 


< 


< 


< 


V2 ||x(a) q s(q) - /i fc +p 
4 miiij Xi(a)si(a) 


4(1 - 26)n t+1 

V26 2 


(1 - 20 ) 


Mfc+l * 


(24) 
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It is easy to check that for 9 < 2+ ^ ~ 0.29289, < 9 holds, therefore, for 9 < 

we have 

||x fc+1 o s fc+1 - Hk+ ie|| < 9fi k+ 1 . 

We now show that (x fc+1 , s fc+1 ) > 0. Let x fc+ 1 (t) = x(a)+fAx and s fc+ 1 (t) = s(a)+fAs. 
Then, x fc+ 1 (0) = x(a) and x fc+ 1 (l) = x fc+1 . Since 

x fc+ 1 (t) o s k+1 (t) = (x(a) + t Ax) o (s(a) + tAs) 

= x(a) o s(a) + t(x(a) o As + s(a) o Ax) + t 2 Ax o As, 

using (17), (15), (24), and the assumption that 9 < 2+ ^ , we have 

||x fc+ 1 (t) o s k+1 (t) - n k+ ie|| 

= ||(1- t) (x(a) o s(a) - /ifc+ie) + t 2 Ax o As|| 

\Z29 2 

< 2(1 — t)9fj, k+ 1 + t 2 ^ _ 2Q ^ k +1 

< (2(1 - t) + t 2 )0/i fc+ i := f(t)9^ k+1 . (25) 

The function /(f) is a monotonical decreasing function of t G [0,1], and /(0) = 2. This 
proves ||x fc+ 1 (f)os fc+ 1 (f) —/i fc+i e|| < 29fi k +i- Therefore, x k+1 (t)s k+1 (t) > (1 — 29)fi k +i > 
0 for all f G [0,1], which means (x fc+ 1 s fc+1 ) > 0. This finishes the proof of part (ii). 

This theorem indicates that the proposed algorithm is well-defined. 


4 Polynomiality 


The analysis follows similar ideas in many existing literatures, such as [16, 22], Let the 
initial point be selected to satisfy 


(x°,s°) G N{9), x* < px°, s* < ps°, (x*, y*, s*) G 5, (26) 

where p > 1. Let ujf and u° be the quality of the initial point which are the “distances” 
from feasibility and optimility given by 


uj f = min{max{||(X°) x (x - x°)||oo, ||(S U ) x (s 


- 0 \-l 


0 \-l/ 


x,y,s 


3 } | Ax = b, A 1 y + s = c}. 

(27) 


and 


u° = min {max{ 


X* T S° S* T X° 


x° T s° ’ x° T s° 


,1} | (l',j',s')£S). 


(28) 


x*,y*,s* 

Let c Up and oj r d be the “ratios” of the feasibility and the total complementarity defined 
by 


u p = 


|| Ax° - b| 

x° T s° 


= 


| A T y° + s° — c| 
x° T s° 


(29a) 

(29b) 
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In view of Lemma 3.1, we have that 


\Ax. k — b11 = cCx fc s fc , 


AV + s 6 — c|| = c^x fe V. 


k?k 


(30a) 

(30b) 


Invoking Lemma 3.3 of [22] for X p = Xd = ( = 1 and (7), we have the following two 
Lemmas [16]. 

Lemma 4.1 Let (x, s) be defined by (7), and = (X fc )^(S fc ) - ^. Then 


max{||(D 


k\- 1 , 


D‘)s||} < || (x‘ o s‘)i || + o/(l + 2V) , (X)8 (31) 

mm fixes'?) 2 


Lemma 4.2 Let (x°, s°) be defined by (26). Then 

or < p, lu° < p. 


(32) 


This leads to the following lemma. 

Lemma 4.3 Let (x, s) be defined by (7). Then, there exists a positive constant C 0 , 
independent of n, such that 

max{||(D /! ) _1 x||, ||(D A )s||} < C 0 ^n(x. k ) T s k . (33) 


Proof: First, it is easy to see 



Since (x fc ,s fc ) G M(9), we have min^xfsf) > (1 — 9)p k = (1 — 9) (x ] n s . Therefore, 


( x ") Tsfc < l n(x k ) T s k ' , 35) 

min i(x k s k )^ \j (1 — 9) 

Substituting (34) and (35) into (31) and using Lemma 4.2 prove (33) with Co = 1 + 

p(l+2p) \ -| I uf(l+2ui°) m 

x/M • 1 














From Lemma 4.3, we can establish several useful inequalities. The following simple 
facts will be used several times. Let u and v be two vectors, then 


lluovll* = £(«,«,)■< p«f) fj>?J = HIW 

(36) 

If u and v satisfy u T v = 0 , then, 


max{||u|| 2 , ||v|| 2 } < ||u|| 2 + ||v|| 2 = ||u + v|| 2 , 

(37) 

and (see [23, Lemma 5.3]) 


1 u O v|| < 2-3||u + v|| 2 . 

(38) 

Lemma 4.4 Let (x, s) and (x, s) be defined by (7) and (8), respectively, 
exist positive constants C\, CV, C 3 , and C 4 , independent of n, such that 

Then, there 

| x 0 s|| < C\n 2 Hk , 

(39) 

| x 0 s|| < C 2 n 4 /i fc , 

(40) 

max{||(D fc ) _ 1 x||, ||(D fc )s||} < C 3 n 2 ^JT k , 

(41) 

max{||x 0 s| , | x 0 s||} < C/pr? [i k 

(42) 


Proof: First, using (36) and Lemma 4.3, we have 

||xos|| = ||(D fc ) _1 x o (D fc )s|| < ||(D A: ) _1 x||||(D / ' )s|| < CQn(x. k ) T s k :=Cin 2 p, k - 
Second, using (38), ( 8 ), (39), and (34), we have 


(43) 


X O S = 


(D fc ) _1 x o (D fc )s|| < 2 - 2 II(D fc ) -1 x 


< 2-3 -2(XS) _ 5(xos) 

n / . . \ 2 n 

- * =**e 


i —1 


i=l 


[XiSiY 


XiSi 


< 2 2 


i EILi 


mm i=lt „. jn 

1 ||x°s || 2 i Cfn 4 nl 
< 2 z— -— < 22 - 1 k 


(1 - 9)nk (1 - 9)p k 

i C 2 n 4 /i k 4 

= 22 := C 2 n fi k . 

Third, using (37), ( 8 ), and (39), we have 


fc \"||2 


(D fc )s 


maxlll(D fc )- 1 x|| 2 , ||(D^s|| 2 i < ||(D fc ) _1 x + (D fc )s|| 2 


—2(XS)“2 (x o s) 


2 ^ dC'fnVfc _ ^4.. 

— \ — Q n ^ k ‘ 


X 

4 


(44) 


(45) 
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Taking square root on both sides proves (41). Finally, using (36), (41), and Lemma 4.3, 
we have 


x o s|| = ||(D fc ) -1 x o (D fe )s|| < ||(D fc ) _ 1 x||||(D fc )s 


< (C 3 n 2 y/pfi)(C 0 ny/JLf) := C A n 3 p k . 

(46) 

Similarly, we can show 


| x 0 s|| < C/pTL 3 Pk- 

(47) 

This finishes the proof. 

1 


Now we are ready to estimate a conservative bound for sin(a). 

Lemma 4.5 Let (x. k ,y k ,s k ) be generated by Algorithm 3.1. Then, sin(a;) obtained in 
Step 3 satisfies the following inequality. 


sin(a) > 


6 


2 Cn 


(48) 


where C = max{l, C* 4 3 , (C\ + 6 * 2 ) 4 }. 


Proof: Let sin(ct) = 7 ^ 6 . In view of (23) and Lemma 4.4, we have 


q(a ) < /Ufc((C*i + C* 2 )n 4 silica) + 2 C* 4 n 3 sin 3 (a;) + 6 1 sin(cr) — 6) := /i fc p(a) 
— pk 


(C 1 + C 2 )9 i 2C 4 0 3 , 9 2 , 

H--r - o 


16C 4 


8C* 3 2Cn 


,'0 4 9 3 9 2 

~ 16 + T + ~2~ * "°- 


Since p(a) is a monotonic function of sin(o(), for all sin(a) < the above inequalities 
hold (the last inequality holds because of 9 < 1). Therefore, for all sin(a) < 7 ^, the 
inequality (15) holds. This finishes the proof. 


Remark 4.1 It is worthwhile to point out that the constant C depends on Cq which 
depends on p, but p is an unknown before we find the solution. Also, we can always find 
a better steplength sin(a) by solving the quartic q(a ) = 0 and the calculation of the roots 
for a quartic polynomial is deterministic, negligible, and independent to n [24, 25], 

Following the standard argument developed in [23], we have the main theorem. 

Theorem 4.1 Let (x fc ,y fc ,s fc ) be generated by Algorithm 3.1 with an initial point given 
by (26). For any e > 0, the algorithm will terminate with (x k ,y k ,s k ) satisfying (14) in 
at most 0{nL ) iterations, where 

L = max{ln((x°) T s°/e), ln(||r°||/e), ln(||r°||/e)}. 

Proof: In view of Lemma 3.1, r^, r k , and Pk decrease at the same rate (1 — sin(o;)) in 
every iteration. Using the Lemma 4.5 and [23, Theorem 3.2] proves the claim. 
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5 Conclusions 


We proposed an infeasible-interior-point algorithm that searches the optimizer along 
an ellipse that approximates the central-path. We showed that the proposed algorithm 
is polynomial and that the polynomial bound is at least as good as the best existing 
bound for infeasible-interior-point algorithms for linear programming. 
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